MemeTector: enforcing deep focus for meme detection

Abstract Image memes and specifically their widely known variation image macros are a special new media type that combines text with images used in social to playfully or subtly express humor, irony, sarcasm even hate. It is important accurately retrieve from better capture the cultural aspects of online phenomena detect potential issues (hate-speech, disinformation). Essentially, background an macro regular easily recognized as such by humans but cumbersome for machine do so due feature map similarity complete macro. Hence, accumulating suitable maps cases can lead deep understanding notion memes. To this end, we propose methodology, called visual part utilization , utilizes instances class initial meme force model concentrate on critical parts characterize meme. Additionally, employ trainable attention mechanism top standard ViT architecture enhance model’s ability focus these make predictions interpretable. Several training test scenarios involving web-scraped controlled presence considered evaluating terms robustness accuracy. The findings indicate light combined sufficient during provides best most robust model, surpassing state art. Source code dataset available at https://github.com/mever-team/memetector .